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Abstract.  This  paper  studies  algorithms  for  decomposing  a  real  image  into  the 
sum  of  cartoon  and  texture  based  on  total  variation  minimization  and  second- 
order  cone  programming  (SOCP).  The  cartoon  is  represented  as  a  function  of 
bounded  variation  while  texture  (and  noise)  is  represented  by  elements  in  the  space 
of  oscillating  functions,  as  proposed  by  Yves  Meyer.  Our  approach  gives  more 
accurate  results  than  those  obtained  previously  by  Vese-Osher’s  approximation 
to  Meyer’s  model,  which  we  also  formulate  and  solve  as  an  SOCP.  The  model  of 
minimizing  total  variation  with  an  Z^-norm  fidelity  term  is  also  considered  and 
empirically  shown  to  achieve  even  better  results  when  there  is  no  noise.  This 
model  is  analyzed  and  shown  to  be  able  to  select  features  of  an  image  according 
to  their  scales. 


1.  Introduction 

Let  /  be  an  observed  image,  which,  in  real  applications,  contains  texture  and/or  noise. 
Texture  is  characterized  as  repeated  and  meaningful  structure  of  small  patterns.  Noise 
is  characterized  as  uncorrelatecl  random  patterns.  The  rest  of  an  image,  which  is  called 
cartoon,  contains  object  hues  and  sharp  edges  (boundaries).  Thus  an  image  /  can  be 
decomposed  as  /  =  u  +  v,  where  u  represents  image  cartoon  and  v  is  texture  and/or 
noise.  A  general  way  to  obtain  this  decomposition  is  to  solve  the  problem 

min  {||s(w)|U  |  \\t(u,  f)\\B  <  a},  (1) 

where  s(-)  and  are  two  functionals  on  appropriate  spaces  and  ||  •  ||^  and  ||  •  lls 

are  norms  (or  semi-norms).  ||  •  \\a  and  s(-)  should  be  chosen  so  that  ||s(u)||a  is  small 
for  regular  signals  u  but  much  bigger  for  irregular  noise  v.  Then,  minimizing  ||s(u)||J4 
is  equivalent  to  regularizing  u  according  to  the  measure  ||s(it)||,4.  A  typical  choice 
for  ||s(u)m  is  f  \Du\p,  where  u  €  BV  (see  Def.  2.1)  and  Du  denotes  the  generalized 
derivative  of  u.  For  p  >  1,  smooth  functions  are  more  regular.  Therefore,  to  keep 
edges  like  object  boundaries  of  /  in  u  (i.e.  allow  discontinuities  in  u),  one  should 
use  p  =  1.  An  adaptive  combination  of  these  semi-norms  is  used  to  keep  sharp  edges 
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while  avoiding  staircase  effects  in  regions  where  the  image  varies  smoothly.  The  fidelity 
term  ||f(u,  /)||s  <  cr  for  some  a  forces  u  to  be  close  to  f.  t(u,f )  is  often  chosen  to 
be  /  —  u  which  is  ideally  the  noise  component  v.  The  choice  of  a  particular  norm 
depends  on  the  application.  In  image  denoising,  a  common  choice  (known  as  the  ROF 
model)  is  \\t(u,  /)||b  =  ||/  —  u\\^,  which  is  small  if  /  —  u  is  noise  or  texture.  In 
deblurring  with  denoising,  for  example,  ||£(u,  /)||b  =  ||/  —  ^4w||l2  is  commonly  used, 
where  A  is  a  blurring  operator.  In  our  application  of  cartoon-texture  decomposition, 
it  is  important  to  choose  a  norm  which  favors  texture  exclusively;  in  other  words,  ||u||_b 
is  small  only  if  v  is  image  texture.  In  both  practice  and  theory,  cartoon  and  texture 
are  not  completely  distinguishable.  It  is  unlikely  that  there  exists  ||£(-,/)||b  so  that 
||t(u,  /)||b  <  oo  for  pure  cartoon  u  but  ||f(it  +  ev,f)\\s  =  oo  for  texture  v  and  any 
e  >  0.  Consequently,  it  is  necessary  to  minimize  a  regularization  term  ||s(u)||J4  while 
imposing  a  constraint  on  the  fidelity  term  || t(u,  /)||s  <  cr.  In  this  paper,  we  show  how 
to  formulate  and  solve  Meyer’s  model  min{J  |Vtt|  s.t.||/  —  «||g  <  cr}  (the  norm  ||  •  ||g 
will  be  defined  in  Section  2  below),  its  approximation,  the  Vese-Osher  (VO)  model, 
and  the  TV+L1  model  min{/  |Vu|  s.t.||/  —  u\\Li  <  cr}  as  second-order  cone  programs 
(SOCPs),  and  compare  these  models  with  one  another. 

Readers  should  keep  in  mind  that  if  ||s(w)||J4  and  ||£(it,  /)||s  are  convex  in  u,  the 
constrained  minimization  problem  min{||s(z()||^  s.t.  ||£(m)||b  <  cr}  is  equivalent  to  its 
Lagrangian  form  min{||s(w)||J4  +  A||f(u,  /)||s},  where  A  is  the  Lagrange  multiplier  of 
the  constraint  ||£(u)||s  <  cr.  The  two  problems  have  the  same  solution  if  A  is  chosen 
equal  to  the  optimal  value  of  the  dual  variable  corresponding  to  the  constraint  of  the 
constrained  problem.  Given  cr  or  A,  we  can  calculate  the  other  value  by  solving  the 
corresponding  problem. 

The  rest  of  the  paper  is  organized  as  follows.  In  Section  2  we  define  certain 
function  spaces  and  norms  on  them  that  are  fundamental  to  the  representation  and 
analysis  of  images.  We  also  present  two  theorems  about  compactness  that  are  useful  in 
our  analysis.  In  Section  3  we  present  the  three  models  for  image-cartoon  decomposition 
mentioned  above.  We  analyze  one  of  them  -  the  TV+L1  model  -  in  Section  4. 
In  particular,  we  relate  the  level  sets  of  the  input  to  the  solution  of  the  TV+L1 
model  using  a  geometric  argument  and  discuss  the  scale-selection  properties  of  this 
model.  In  Section  5  we  give  the  SOCP  formulations  of  the  three  models  presented 
in  Section  3,  after  introducing  some  notation  and  background  for  SOCP.  Numerical 
results  illustrating  properties  of  the  models  are  given  in  Section  6. 

2.  The  BV  and  G  spaces  and  norms 

In  this  section,  we  formally  define  the  Banach  space  BV  of  functions  with  bounded 
variation  and  the  Banach  space  G,  which  is  dual  to  a  subspace  of  BV,  and  norms 
defined  in  these  spaces. 

Definition  2.1  [28]  Let  u  £  L1,  and  define 

\\Du\\  :=  sup  |y  udiv(g)dx  :  g  £  Gg(K";]R"),  \g{x)\p  <  1  for  all  x  £  M”| , 

and  ||u||bv  :=  HwIIl1  +  ||Lhi||,  where  Cg(Rn;Rn)  denotes  the  set  of  continuously 
differentiable  vector-valued  functions  that  vanish  at  infinity.  The  Banach  space  of 
functions  with  bounded  variation  is  defined  as 
BV  =  {u  £  L1  :  ||w||  w  <  oo}  , 
and  is  equipped  with  the  ||  •  || BV-norm. 
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BV (f2)  with  D  being  a  bounded  open  domain  is  defined  analogously  to  BV  with  L 1 
and  Cq(R";R")  replaced  by  L1( Cl)  and  Cg  (fl;  R"),  respectively. 

To  help  understand  this  definition,  let  us  consider  the  case  that  u  G  C1;  then  Du 
is  the  usual  derivative  vector  Vit,  i.e. , 

\\Du\\  =  f  \Vuy  =  [ 

J  o  Jo. 

since 

J  udiv(g)  dx  =  —  J  g-\/udx.  (2) 


.  du  du  . 

dxi  ’  ’  dxn 


dx 


Also  noting  that  Cg(R”;R”)  is  dense  in  Co(R";R”),  we  can  define  the  following 
functional  on  C0(Rn;R"):  [14] 

Lu(g)  =  /  udiv(g)dx:=  lim  /  ud\v{gi)dx  (3) 

J  i^ooj 

for  g  G  Co(R";R")  where  G  Cg(Rra;R")  uniformly  converges  to  g  (thanks  to 
1 1 it 1 1 sy  <  oo,  the  limit  does  not  depend  on  the  choice  of  sequence). 

More  generally,  if  u  is  in  the  Sobolev  space  IV1,1  (Rn),  then  Du  is  the  generalized 
derivative  of  u.  Therefore,  from  the  definition  (3)  and  using  integration  by  parts,  we 
have 

u  G  BV  n  W1,1(R")  sup  f  Du  ■  g  <  oo.  (4) 

geC0(K";K")  |fl(*)|,2<l  •/ 

We  can  also  see  from  (4)  that  each  u  defines  a  bounded  linear  functional  Lu(g) 
on  Co(Rn;M")  [14].  Using  the  Riesz  representation  theorem  (also  referred  to  as  the 
Riesz-Markov  theorem)  on  the  isomorphism  between  the  dual  of  Co(R";R")  and  the 
set  of  bounded  vector  Radon  measures,  we  immediately  have  the  following  equivalent 
and  often  used  definition: 


BV  =  {'«  :  Du  is  a  bounded  Radon  vector  measure  on  M"}  . 

When  Du  is  considered  a  measure,  ||Du||  over  a  set  f 1  C  R”  equals  the  total  variation 
of  Du  as  the  Borel  positive  measure  over  fi.  This  is  given  by 

{n  n 

\\Du(Ei)\\  :  Ei  C  ft,  Ei  s  are  disjoint  Borel  sets 

i—l  i=l 

where  the  Borel  sets  are  the  cr-algebra  generated  by  the  open  sets  in  R".  This  is 
true  because  each  g  €  Co(Rn;R")  such  that  ||<f||;2  <  1  is  the  limit  of  a  series  of 
[—1,  l]-valued  vector  functions  that  are  piecewise  constant  on  Borel  sets. 

In  the  dual  space  of  Co(Rn;Rn)  we  define  weak-*  convergence  of  Dun  to  Dn  as 


lim 

n—>  oo 


Dun  ■  g  = 


Du  ■  g, 


for  all  g  €  C0(Rn;Rn). 

Sets  in  R"  with  finite  perimeter  are  often  referred  to  as  BV  sets.  The  perimeter 
of  a  set  S  is  defined  as  follows: 


Per(S)  :=  ||D1S||  =  sup 


di v(g)dx  :  g  €  Cq(R";R"),  \g(x)\p  <  1,V  x  G  R 


where  Is  is  the  indicator  function  of  S. 
Next,  we  define  the  space  G  [18]. 


(5) 
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Definition  2.2  Let  G  denote  the  Banach  space  consisting  of  all  generalized  functions 
v(x)  defined  on  IR”  that  can  be  written  as 

v  =  div(g),  j  =  fe]i=1 . „er(i";r),  (6) 

and  equipped  with  the  norm  ||i>]|g  defined  as  the  infimum  of  all  L°°  norms 
of  the  functions  \g(x)\i2  over  all  decompositions  (6)  of  v.  In  short,  ||w||g  = 
inf{||  \g(x)\i2  ||L=o  :  v  =  div(g)}. 

G  is  the  dual  of  the  closed  subspace  BV  of  BV,  where  BV  :=  {u  G  BV  :  \Du\  €  L1} 
[18].  We  note  that  finite  difference  approximations  to  functions  in  BV  and  in  BV  are 
the  same.  For  the  definition  and  properties  of  G(fl),  see  [6]. 

An  immediate  result  of  Definition  2.2  is 

J  uv  —  J  mV  •  g  =  -  J  Du  ■  g  <  \\Du\\  ||w||g,  (7) 

holding  for  any  u  G  BV  with  a  compact  support  and  v  G  G.  We  say  (m,  v)  is  an 
extremal  pair  if  (7)  holds  with  equality. 

The  following  lemma  [5,  15,  21]  is  very  useful  in  many  proofs: 

Lemma  2.3  Let  q  G  [l,oo)  and  u  G  BV  H  Lq .  Then  there  exists  a  sequence  {m;}/6n 
in  Cq°  satisfying 

ui  — >  u  in  Lq  and  ||Dm;||  — >  ||Dm||. 

The  same  also  holds  for  u  G  BV(Vl). 

For  a  proof  of  this  lemma,  see  [21].  We  note  that  if  S2  :=  spt(u),  the  support  of  u, 
is  bounded,  then  Lq(D)  C  L1( H)  holds  for  any  q  >  1;  hence,  one  only  requires  u  G  BV 
(by  definition  BV(D)  C  L1(fl))  for  the  result  in  Lemma  2.3  to  hold. 

Theorem  2.4  (See  [21,  28]  for  related  results)  If  a  sequence  defined  in  BV 

satisfies  supi  ||Dm,  ||  <  +oo,  then  it  has  a  subsequence  that  weakly  converges  in  both 
j_n/{n- 1)  an( i  gy  t0  u  £  p$y  Moreover,  weak  lower  semi- continuity  holds  for  this 
sequence: 

||Dm||  <  lim inf j||Z?Mj||. 

Proof  Let  us  apply  the  previous  lemma  with  q  =  n/(n  —  1)  for  each  up.  there  exist  a 
sequence  {m.^}  in  Cq°  such  that  u,;,;  — >  m,  strongly  in  and  UDMj^H  — »  ||DMj|| 

as  l  — >  oo.  First  we  prove  HMjH^n/fn-i)  is  bounded: 

\\Ui\\ Ln/(n-l)  =  SUp  /  WUi 

wecg°  ,\\w\\Ln<l  J 

=  sup  lim  /  WUi  i 

loeCfMMUn^i  l^°°J 

<  liminfJ_>0O||ui)i||in/(n-.i) 

<  lim  \\Duitl\\ 

l— >oo 

=  WDuil 

where  the  first  equality  follows  from  the  fact  that  Cg°(fl)  is  dense  in  Ln  =  (L™/!”-1))*, 
the  second  equality  holds  because  strong  convergence  implies  weak  convergence  (using 
Holder’s  inequality),  and  the  second  inequality  follows  the  Gagliardo-Nirenberg- 
Sobolev  inequality  applied  to  upi  G  Cq°: 

II/||l"/("-D  <  J  |V/|,r  for  any  r  >  1. 
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From  sup,;  ||Lh/j||s„  <  +oo,  the  uf  s  are  uniformly  bounded  in  Ln^n~l')  and,  hence, 
there  exists  a  u  £  that  is  the  weak  limit  of  some  subsequence  of  {iq}iSN.  Let 

us  continue  to  write  the  subsequence  as  {tqjieN-  Then  from 


| Du||  =  sup  /  udiv(g) 

9dC q ,  \gUs<lJ 

=  sup  lim  /  Uidiv(g) 
gecv  \g\is<ii^+°°J 

<  lim  infd| Du, ||, 


we  have  u  £  BV.  Moreover,  noting  that  the  second  equality  still  holds  for  any  g  £  Cq 
without  taking  the  supreme  over  g,  we  have  {rq}  also  weakly  converges  to  u  in  BV. 
In  other  words,  {Du{\  converges  to  Du  as  measures.  I 


This  theorem  proves  the  so-called  relatively  weakly  compact  property.  For  Lp  (1  < 
p  <  n/(n  —  1),  the  next  theorem  gives  a  stronger  result: 

Theorem  2.5  (See  [1,  15])  Let  S  be  a  BV-bounded  set  of  functions  defined  on  D 
bounded.  Then  S  is  relatively  compact  in  Lp  for  1  <  p  <  n/(n  —  1). 

Theorems  2.4  and  2.5  are  used  in  Section  3.3  to  prove  that  the  solutions  of  the 
perturbed  TV+L1  model  converge  to  the  solution  of  the  TV+L1  model. 


3.  Three  cartoon-texture  decomposition  models 

In  the  rest  of  the  paper,  we  assume  the  input  image  /  has  a  compact  support  contained 
in  a  bounded  convex  open  set  with  a  C 1  boundary. 

3.1.  Meyer’s  model 

To  extract  cartoon  u  in  the  BV  space  and  texture  and/or  noise  v  as  an  oscillating 
function,  Meyer  [18]  proposed  the  following  model: 

inRf  {  /  lv«l  :  IM|g  <  cr,  /  =  u  +  n}.  (8) 

u£BV  J 

As  we  have  pointed  out  in  Section  2,  G  is  the  dual  space  of  BV ,  a  subspace  of  BV. 
So  G  is  closely  connected  to  BV.  Meyer  gave  a  few  examples  in  [18]  illustrating  the 
appropriateness  of  modeling  oscillating  patterns  by  functions  in  G. 

Unfortunately,  it  is  not  possible  to  write  down  Euler-Lagrange  equations  for 
the  Lagrangian  form  of  Meyer’s  model,  and  hence,  use  partial  differential  equation 
methods  to  solve  it.  Alternatively,  several  models  [6,  7,  22,  26]  have  been  proposed  to 
solve  (8)  approximately.  The  Vese-Osher  (VO)  model  [26]  approximates  ||  \g(x)\i2  ||x,°o 
by  ||  \g(x)\p  || lp,  with  1  <  p  <  oo.  It  is  described  in  Section  3.2.  The  Osher-Sole- 
Vese  model  [22]  replaces  ||w||g  by  the  Hilbert  functional  ||n||^_i.  The  more  recent 
A2BC  model  [6,  8,  7],  inspired  by  Chambolle’s  projection  algorithm  [11],  minimizes 
TV{u)  +  A||/  —  u  —  i>||2  for  (u,v)  €  BV  x  {v  £  G  :  ||u||g  <  p}-  A  similar  projection 
algorithm  [9]  is  also  used  to  approximately  solve  the  TV+L1  model  described  in 
Subsection  3.3  below.  In  Section  5  we  present  an  SOCP-based  optimization  model 
to  solve  (8)  exactly  (i.e. ,  without  any  approximation  and  regularization  applied  to 
/  |Vu|  and  ||u||g  except  for  the  use  of  finite  differences). 
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3.2.  The  Vese-Osher  model 

Motivated  by  the  definition  of  the  L°°  norm  of  \g(x)\ii  as  the  limit 

II  W\p  II =  lim  II  W\p  I \Lp  - 


p — >-oo 


(9) 


Vese  and  Osher  [26]  proposed  the  following  approximation  to  Meyer’s  model  (8): 

I  i/p 


\g\\ 


}-  (io) 


inf  {VOp{u,g):=  f  |Vu|  +  A  f  \f-u-dW(g)\2  +  g,  f 

u<EBV,g£C'(  K";K")  J  J  [J 

In  ]R2,  minimizing  VOp  with  respect  to  u,g  =  (51,32)  yields  the  associated  Euler- 
Lagrange  equations: 


u  =  f  -  digi  -  d2g2  +  ^  div 

i-p 


Vu 
jVuj 

P-2 


(11) 


\Jg\  +  g2\\^j  {yjgl+gi^  g1=2X[d1(u- f)  +  d^1g1  +  d^2g2\,  (12) 

V  (\\\Jgl  +  gl\\^j  Jg\  +  gl ^  gi  =  2A  [d2(u  -  /)  +  +  d%2g2\  .  (13) 

In  [26] ,  the  authors  solve  the  above  system  of  partial  differential  equations  for  different 
values  of  p,  with  1  <  p  <  10,  via  a  sequential  descent  approach  and  report  that  they 
give  very  similar  numerical  results. 


3.3.  The  TV+L 1  model 

In  [3,  10,  20]  the  square  of  the  L 2  norm  of  /  —  u  in  the  fidelity  term  in  the  original 
ROF  model  (min {TV (u)  +  X\\f  —  u||22})  is  replaced  by  the  Ll  norm  of  /  —  u,  which 
yields  the  following  model: 

min  TVLl\(u)  =  min  [  |Vtt|  +  A  [ \f  —  u\.  (14) 

u€BV  u€BV  J q  J 

Although  this  model  appears  to  be  simpler  than  the  Meyer  and  VO  models,  it  has 
recently  been  shown  to  be  extremely  powerful  in  background  correction  applications 
[12,  27].  It  also  has  the  important  property  of  being  able  to  separate  out  features  of 
a  certain  scale  in  an  image  as  we  shall  show  in  the  next  section. 


4.  Analysis  of  the  TV+L1  model 

We  now  analyze  the  TV+A1  model  assuming  that  the  input  image  /  is  defined  in  L1 
and  has  a  compact  support  spt(/)  contained  in  a  bounded  convex  open  set  with  a 
C 1  boundary.  Recall  that  BV(Cl)  and  ||Z?it||  limited  to  f 1  are  defined  in  analogy  with 
BV  and  ||Dii||  where 

||Du||n  :=  SUP  |y  udiv(<?)  <  00 ,<7  €  Co(fl;K”),  \g(x)\i2  <  1 


and 


BV(Q)  =  {«£  L1(Q)  :  ||u||w(n)  <  00}. 
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Consider  the  approximate  TV+L1  model  in  which  a  perturbation  e  has  been  added 
to  the  fidelity  term  ||/  —  u\\Li  to  make  it  differentiable: 

min  TVLlxAu)  :=  min  [  |Vw|  +  A  [  \J (f  —  u )2  +  e.  (15) 

U&BV(Q)  ’  ueBV{Q.)  Jq 

Since  TVLlAe(w)  is  strictly  convex,  problem  (15)  has  a  unique  solution  uA>e(/)  (or, 

U\>e). 

Lemma  4.1  Sets  Go  :=  {v  :  v  =  div(g),  g  £  Cg(n;K”),  ||  \g(x)\p  ||loo  <  1  }  and 
BV o  :=  {u  €  L1(fl)  :  \\Du\\  <  R,  ||u||li  <  II/IIl1}  C  BV(fl),  where  R  is  given,  are 
convex.  Moreover,  BVq  is  compact  in  Ll . 

Proof  Suppose  vg  and  Vh  are  in  G0.  There  exist  g,h  £  Cq  (Q:  M”)  satisfying 

vg  =  div(g),  vh  =  div(ft),  ||  \g\p  ||too  <  1,  ||  \h\t2  ||Loo  <  1. 

For  any  A  £  [0, 1],  we  have  (Minkowski  inequality) 

II  1-^5  +  (1  —  X)h\e  Ik00  —  II  Mt)\i2  +  (1  ~  A)|/r|i2  ||l~ 

<  A||  \g\12  ll^oo  +  (1  —  A) ||  \h\p  ||z,oo 

<  A  +  (1  —  A)  =  1. 

This  means  ||Aug  +  (l  — A)u/1||g  <  1;  consequently,  Av9  +  (1  —  X)vh  £  Go-  The  convexity 
of  BV o  can  be  proved  analogously  from  its  definition.  The  compactness  of  BV o  in  L1 
is  a  direct  result  of  Theorem  2.5.  ■ 

Using  this  lemma,  we  shall  analyze  an  important  property  of  the  texture  output 
Vx,e  of  the  approximate  TV+L1  model  (15)  using  the  G-norm  in  Theorem  4.3  below. 
At  the  end  of  this  subsection,  we  shall  show  that  Vx,e  converges  to  vx  as  e  goes  to  0. 
We  first  state  the  generalized  minimax  theorem,  which  is  used  in  the  proof  of  Theorem 

4.3. 

Theorem  4.2  (Generalized  Minimax  Theorem  [13,  25])  Let  K  be  a  compact 
convex  subset  of  a  Hausdorff  topological  vector  space  X,  C  be  a  convex  subset  of  a 
vector  space  Y,  and  f  be  a  real-valued  functional  defined  on  K  x  C  which  is  (1) 
convex  and  lower-semicontinuous  in  x  for  each  y,  and  (2)  concave  in  y  for  each  x. 
Then 

inf  sup  /( x,  y)  =  sup  inf  f(x,  y). 
x&R  yec  yGCx^K 

Theorem  4.3  The  solution  ux,e{=  f  —  Vx,e)  €  BV{VL)  of  the  approximate  TV+L 1 
model  satisfies 

||signe(vA,e)||G  <  1/A, 

where  signe(-)  is  defined  point-wise  by  signc (g) (a;)  :=  g{x) / x/\g(x)\2  +  e  for  any 
function  g. 

A  proof  for  more  general  cases  can  be  found  in  [21].  We  give  a  short  proof  below 
based  on  Theorem  4.2.  A  similar  approach  is  also  used  in  [17]  to  derive  the  G-norm 
related  properties  for  the  ROF  model  [24]. 
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Proof  Let  R  (in  the  definition  of  BVq  in  Lemma  4.1)  be  large  enough,  and  consider 
the  functional  L  :  BVq  x  Go  — * ►  R: 

L£(w,uj)  =  /  uw  +  A >/ (/  —  u )2  +  e. 

Jn 

Define  Pe(u)  =  supu,gGo  Le(u,  w)  and  De(w )  =  inf^gsvb  Le(u,  w).  L(u,w)  is  convex 
and  lower  semi-continuous  in  u,  and  is  linear  (hence  concave)  in  w. 

Since  Go  is  complete  w.r.t.  ||  •  ||G,  there  exists  an  optimal  wXje(u)  satisfying 
Pe(u)  =  Le(u,  wAj£(it))  for  each  u  £  BVq.  On  the  other  hand,  by  applying  Theorems 
2.4  and  2.5,  we  have  an  optimal  uAi£  £  BVq  that  minimizes  Pe(u). 

The  obtainability  of  optimizers  and  Lemma  4.1  allow  us  to  apply  Theorem  4.2  to 
Le(u,w):  there  exists  an  optimal  solution  pair  (kAi£,  u>AjC)  £  BVq  x  Gq  such  that 


De{w  x,e)  =  Le{uX,e,WX,e)  =  Pe{uX,e)- 
The  first  equation  indicates  dLe(u,  wXt€)/du\u=Ux  e  =  0,  and  this  gives 

W\e  =  A 

where  uAj£  =  /  -  ux>e.  Therefore,  ||sign£(uA;£)||G  <  1/A. 


v\,e 


(16) 

(17) 


Corollary  4.4  If  ||signe(/)||G  <  1/A,  ux^  =  0  is  the  solution  of  the  approximate 
TV+L 1  model. 


Proof  Let  wX}6  =  A  sign e(/)  and  uAjC  =  /.  Noting  that  uA,e  =  /  —  uAjC  =  0,  we  have 
D0{wX}e)  =  L0(uXi€,wXt<i)  =  P0(wA>e). 

The  result  then  follows  from  the  optimality  of  the  saddle  point  (uA>e,  wx,e).  ■ 

Corollary  4.5  If  ||signe(/) ||G  >  1/A,  then  there  exists  an  optimal  solution  ux,e 
satisfying 

•  l|signe(uAi£)||G  =  1/A; 

•  /  uA>e  signe(uAie)  =  ||-DiiA)€||/A,  i.e.,  uAje  and  signe(uA,e)  form  an  extremal  pair. 

Proof  Since  ||signe(/)||G  >  1/A  but  ||signe(uAi£)||G  <  1/A  by  Theorem  4.3,  we  must 
have  mAi£  ^  0.  Then,  we  have  ||rcA,e||G  =  1  from  the  second  equation  in  (16). 
If  follows  from  (17)  that  ||sign£(uAjfr)||G  =  1/A.  The  other  property  follows  from 
fQ  uXjewx,e  =  supweGo  fn  ux>ew  =  ||DuA,€||  and  the  equation  wAj£  =  Asign£(uAj£).  ■ 

Theorem  4.6  Assuming  the  TV+L 1  model  (If)  using  parameter  A  has  a  unique 
solution  ux,  then  the  solution  of  approximate  TV+L 1  model  (15)  using  the  same 
parameter  A  satisfies 

lim  ||uA  £  -  uA||Li  =  0,  lim  ||uA>e  -  uA||x,i  =  0. 

eiO+  e|0+ 

Before  proving  the  theorem  we  note  that  the  optimal  solution  of  the  TV+L1  model  (14) 
is  not  unique  for  a  certain  set  of  A’s  as  the  objective  function  TVLlx(u)  is  not  strictly 
convex.  Examples  of  nonunique  solutions  can  be  found  in  [10,  27].  However,  Chan 
and  Esedoglu  [10]  proved  that  the  set  of  such  A’s  is  at  most  countable.  Consequently, 
the  TV+L1  model  (14)  has  a  unique  solution  for  almost  all  A’s  w.r.t.  the  Lebesgue 
measure. 

Proof  of  Theorem  f.6. 
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Proof  of  Theorem  4.6  Noticing  that  \Jt  +  e  <  \Jt  +  i/e,  for  t,  e  >  0,  we  have,  for 
all  positive  e  less  than  a  given  e0> 


TVLlx,e(ux,e)  <  TVLlx,e(ux)  <  TVLlx(ux)  +  sfe.  (18) 

From  this  we  conclude  that  TVLlx>t(ux^),  and  thus  ||DuA,e||,  are  bounded.  Using 
the  fact  that  fi  is  bounded  (/  has  compact  support)  and  applying  Theorem  2.5 
with  p  =  1  and  n  =  2,  we  conclude  that  there  exists  u  £  DU(fi)  such  that 
Hindoo  ||itA,ei  —  ulU1  =  0  with  lim^oo  e*  =  0.  The  optimality  of  u  follows  from 


TVLlx(u)  =  \\Du\\+X  J  \u- f\ 

=  || Du||  +  A  lim  /  J (ux  u  —  /)2  +  €i  (dominant  convergence) 
i—*  oo  J  V 

<  liminf  \\Dux^.  ||  +  A  /  J (ux  ti  —  f)2  +  e,  (lower  semi-continuity) 

i—> oo  ’  %  J  V  '  % 


=  liminf  TVLlXt<ii(uX}ei) 

i—>  oo 

<TUL1a(ua)  (by  (18)). 


By  the  uniqueness  of  ux,  limc^o+  ||MA,e  —  maIIl1  =  0.  Since  f  £  L1  and  hence 
vx  =  f  -  ux  £  L1,  we  also  have  lime^0+  ||fA,e  -  vx\\Li  =0.  ■ 


4-1-  TV+L 1  Geometry 


It  is  well  know  that  the  G'-norm  (Def.  2.2)  is  a  good  measure  of  the  oscillation  of 
functions.  Using  G- value  [23],  which  is  an  extension  of  Meyer’s  G-norm,  we  are  able  to 
fully  characterize  the  texture  output  of  the  TV+L1  model  for  a  given  parameter  A.  In 
addition  to  cartoon-texture  decomposition,  the  TV+L1  model  can  be  used  to  separate 
large-scale  and  small-scale  features.  In  these  applications,  we  are  often  interested  in  an 
appropriate  A  that  will  allow  us  to  extract  geometric  features  of  a  given  scale  measured 
by  the  G-value  of  the  feature  set.  For  general  input,  the  TV+L1  model,  which  has 
only  one  scalar  parameter  A,  returns  images  combining  many  features.  Therefore,  we 
are  interested  in  determining  a  A  that  gives  the  whole  targeted  features  with  the  least 
unwanted  features  in  the  output. 

For  simplicity,  we  assume  U  =  K2  in  this  section.  Our  analysis  starts  with  the 
decomposition  of  /  using  level  sets  and  relies  on  the  co-area  formula  (19)  [15]  and 
“layer  cake”  formula  (20)  [10],  below.  We  first  need  the  following: 

Definition  4.7  The  (upper)  level  set  of  a  function  g  at  level  fi  is  defined  by 
U{g,p)  ■■=  {x  £  Dom(g)  :  g(x)  >  fi}. 


The  co-area  formula  for  functions  of  bounded  variation  is 


Per(f7 (it,  p))  dp, 


(19) 


where  Per(-)  is  the  perimeter  function  defined  in  (5).  Using  (19),  Chan  and  Esedoglu 
[10]  showed  that  the  TVL1X  functional  can  represented  as  an  integral  over  the 
perimeter  and  weighted  areas  of  certain  level  sets  by  the  following  “layer  cake”  formula: 

nOO 

TVLlx(u)=  /  (Per(D(u,/z))  +  \\U(u,  p)\U(f,  p)\  +  X\U(f,  p)\U(u,  p)\)  dp,  (20) 


where  |£|  for  a  set  S  returns  the  area  of  S.  Therefore,  an  optimal  solution  ux  to  the 
TV+L1  model  can  be  obtained  by  minimizing  the  right-hand  side  of  (20). 
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Let  us  fix  A  and  focus  on  the  integrand  of  the  above  functional  and  introduce  the 
following  notation: 

C(r, E)  :=  Per(E)  +  A|E\r|  +  A|r\E|,  (21) 

where  T  and  E  are  sets  with  bounded  perimeters  in  R2. 

We  are  interested  in  determining  whether  we  can  obtain  u\  and  its  geometric 
properties  by  exchanging  the  minimum  and  the  integral  operators  and  solving 

min  C(r,  E)  (22) 

for  every  T  =  U(f,p)  where  /i  G  [—00,00).  Let  E/iM  denote  a  solution  of  (22)  for 
T  =  U(f,p).  For  the  existence  of  a  u  satisfying  U{u,p)  =  E/iM  for  all  p,  we  need 
EyiA1 1  I)  EyiAt2  for  any  pi  <  p2,  since  it  is  clear  from  Definition  4.7  that  for  a  given 
g,  U(g,p)  is  monotonically  decreasing  with  respect  to  p.  This  result  is  proved  in 
Theorem  4.12. 

Lemma  4.8  ([4],  Prop.  3.38)  If  two  arbitrary  sets  S 1  and  S2  have  finite  perimeter, 
then  S\  fl  S2  and  Si  U  S2  have  finite  perimeter  and 

Per(S'i  fl  1S2)  +  Per(5i  U  S2)  <  Per(Si)  +  Per^)-  (23) 

The  above  lemma  is  used  in  the  proof  of  Lemma  4.9  below. 

Notation:  in  the  rest  of  this  section,  we  assume  that  Ti  and  r2  are  two  sets  with 
Ti  D  r2  and  the  sets  Ei  and  E2  are  solutions  of  (22)  for  T  =  Ti  and  T  =  r2, 
respectively.  For  arbitrary  sets  T  and  E,  we  write  T  +  E  instead  ofruEifrnE  =  0 
holds,  and  write  T  —  E  instead  of  T\E  if  E  C  T  holds. 

Lemma  4.9  Defining  Ti,  r2,  Ei,  and  E2  as  in  the  above  notation,  we  have 
C{TU  E2)  -  C'(r1,S2  n  Ei)  >  C(Ti,  Er  +  (Ea^,))  -  C^,  E,).  (24) 

Lemma  4.10  Defining  Ti,  r2,  Ei,  and  E2  as  in  the  above  notation,  we  have 

C(T2, e2)  -  c(r2, e2  n  sx)  >  c(rl5  s2)  -  c(r1} s2  n  e,).  (25) 

To  demonstrate  the  use  of  Lemmas  4.9  and  4.10,  we  first  present  and  prove  the 
following  theorem: 

Theorem  4.11  Let  r1;  r2,  Elf  and  E2  be  defined  as  in  the  above  notation.  If  either 
one  or  both  of  Ei  and  E2  are  unique  minimizers,  then  Ei  D  E2;  otherwise,  i.e.,  both 
are  not  unique  minimizers,  Ei  D  E2  may  not  hold,  but  in  this  case ,  Ei  U  E2  is  a 
minimizer  of  (22)  for  T  =  Ti.  Therefore,  there  always  exists  a  solution  of  (22)  for 
r  =  Ti  that  is  a  superset  of  any  minimizer  of  (22)  for  T  =  T2. 

Proof  Combining  Lemmas  4.9  and  4.10,  we  have 

o  >  c(r2, e2)  -  c(r2,E2  n  sx)  >  c(rl5  s2)  -  c(r1; e2  n  sx) 

>  C(r1;  Ei  +  (E2\Er))  -  C(r!,  Ei) 

>  o. 

Clearly,  all  the  inequalities  hold  as  the  equalities.  Consequently,  Ei  +  (E2\Ei)  = 
Ei  U  E2  minimizes  C(ri,-).  If  Ei  is  the  unique  minimizer  of  C'(Fi,-),  then 
Ei  +  (E2\Ei)  =  Ei  which  gives  Ei  D  E2.  If  E2  is  the  unique  minimizer  of  C'(r2,-), 
then  E2  =  E2  fl  Ei  which  also  gives  Ei  D  E2.  ■ 
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This  theorem  serves  as  a  foundation  for  Theorem  4.12.  Next,  we  give  the  proofs 


of  Lemmas  4.9  and  4.10. 

Proof  (Lemma  4.9)  By  Definition  4.7, 

C(rlt  E2)  -  C(r1}  E2  n  Ei)  =  (Per(S2)  -  Per(S2  n  Ei))  (26) 

+  A(|E2\r1|-|(E2nE1)\r1|)  (27) 

+  A(|T1\E2|-|r1\(E2nE1)|),  (28) 

C(rlt  Ei  +  (E2\Ei))  -  C(T1;  EO  =  (Per(E1  +  (E2\Ei))  -  Per^))  (29) 

+  A(|(EX  +  (S2\E1))\r1|  -  |E ATil)  (30) 

+  A(|r1\(E1  +  (E2\E1))|-|r1\E1|).  (31) 


To  compare  (26)  with  (29),  we  let  Si  :=  Ei  and  S2  :=  S2  in  (23);  hence, 
(S*i  D  S2  =  S2  fl  Ei  and  ^  U  52  =  Ej  U  E2  =  Ei  +  (E2\Ei).  From  Lemma  4.8, 
we  have 

Per(E2  D  Ei)  +  Per(Ei  +  (E2\Ei))  <  Per(Ei)  +  Per(E2), 
which  is  equivalent  to 

Per(S2)  -  Per(E2  n  EJ  >  Per(E!  +  (E2\E!))  -  Per(Ei). 

Next,  we  show  that  (27)  equals  (30). 

I s2\r ! |  -  |(E2  n  Si)\r!|  =  |((e2  n  so  +  (e2\e1))\t1|  -  |(e2  n  e 1^1 
=  |(e2\e1)\t1|  +  |(e2  n s OYTil  -  |(s2  n (•.•  (e2  n so  n  (e2\Ei)  =  0) 

=  |(E2\E1)\r1|  +  |s1\r1|-|E1\r1| 

=  |(Si  +  (s2\s1))\r1|  -  lEATii  (•.■  (e2\E!)  n  e,  =  0). 

The  last  step  is  to  show  that  the  remaining  terms  (28)  and  (31)  are  equal. 

ITAEal  -  |r1\(S2  nEOI  =  |T!  nEdl  -  |T!  n  (EaTlEOI 
=  -|r1n((E2nSi)-E2)|;  (•.•  E2  c  (e2  n  Ei)  ) 

=  —  |Ti  n  (s2\E! ) | ;  (•.■  E2\E!  =  e2  —  e2  n  sx  =  (s2  n  so  —  s2 ) 

=  -|Ti  n(Ed\s^)| 

=  — |Ti  n(Ed-  (EdnE^)| 

=  |r i  n  (Si  u  e2)|  —  |Ti  n  Ei|;  (■.■  Si  d  Ei  n  s2  =  (Ei  u  e2)  ) 

=  |r1n(E1  +  E2\s1)|-|r1nE7| 

=  |r1\(E1  +  s2\s1)|-|r1\E1|. 


Proof  (Lemma  4.10)  First,  we  expand  the  left-hand  side  of  (25)  as  follows: 

C(r2,  S2)  -  C(r2,  E2  n  Ei)  =  Per(S2)  -  Per(S2  n  SO  (32) 

+  A(|s2\r2|-|(E2ns1)\r2|)  (33) 

+  A(|r2\s2|-|r2\(E2ns1)|).  (34) 
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The  right-hand  side  of  (25)  is  expanded  in  (26)-(28).  As  (32)  is  identical  to  (26),  we 
only  need  to  compare  (33)  and  (34)  to  (27)  and  (28),  respectively.  In  fact, 

|s2\r2|  -  |(s2  n  s1)\r2|  =  |(e2\(e2  n  E^VM 

=  KEAEr)^! 

>  |(s2\E1)\r1|  (■.'  Ti  d  r2) 

=  |(E2\(E2ns1))\r1| 

=  |E2\r1|-|(E2ns1)\r1|, 


|r2\E2|  -  |r2\(s2  n Er)|  =  |r2  n  e^|  -  |r2  n  (e2  n  ex)| 
=  -|r2n((E2ns1)-Ei)|  (vAc(s2ns1)) 

>  -|r1n((E2nE1)-E^)|  (-.t^  r2) 

=  ir.ns^l-  |r1n(s2nE1)| 

=  |r1\s2|  -  |r1\(E2nE1)|. 


We  are  now  ready  to  give  the  main  result  of  this  section: 

Theorem  4.12  Suppose  that  f  £  BV  has  essential  infimum  p o-  Let  function  u*  be 
defined  point-wise  by 

nOO 

u*{  x):=p0  +  1  sf!li(x)dp,,  (35) 

I  no 

where  E ftfl  is  the  solution  of  (22)  for  T  =  U(f,p)  that  satisfies  E/iMl  D  S/jAt2  for 
any  pi  <  p2,  i.e.,  E ftfl  is  monotonically  decreasing  with  respect  to  p.  Then  u*  is  an 
optimal  solution  of  the  TV+L 1  model  (If). 


Proof  First,  Theorem  4.11  guarantees  that  there  exists  E/  M  as  the  solution  of  (22) 
for  T  =  U(f,p)  that  is  monotonically  decreasing  with  respect  to  p.  Second,  from  (35) 
and  the  monotonicity  of  E f  ^  and  ls/  with  respect  to  p,  we  have 
/»00 

U(u*,p)  =  {x  :  /  l^f  ii(x)dp  >  p-  po}  =  {x  :  lE/i/t(a;)  =  1}  =  E/>M, 

J  Ho 

for  p  £  [po ,  oo) .  In  addition,  for  p  £  (— oo,/io),  U(u*,p)  =  U(f,p)  =  R2. 
C(R2,R2)  =  0.  Therefore,  for  each  p  £  (—00,00),  U(u*,p)  minimizes  the  integrand 
in  (20).  Finally,  we  have 


/OO 

(Per [U{u\p))  +  A  | U(u*,p)\U(f,  p) \  +  A  | U(f,  p)\U(u*,p)\)  dp 

-00 


< 


(Per([/(u,  p))  +  A  \U(u,p)\U(f,  p)\  +  A  | U(f,  p)\U(u,p)\)  dp 


=  TVLlx{u), 


for  any  u  £  BV.  Consequently,  u*  is  an  optimal  solution  of  the  TV+L1  model  (14). 


Next,  we  use  results  from  [23]  given  below  to  illustrate  the  implications  of 
Theorem  4.12. 
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Definition  4.13  (G-value) [23]  Let  +  :  R2  — >  2K  be  a  set-valued  function  (also  called 
as  a  multifunction  and  a  set-valued  map)  that  is  measurable  in  the  sense  that  'F~1(>S') 
is  Lebesgue  measurable  for  every  open  set  ScM.  We  do  not  distinguish  4'  between  a 
set-v alued  function  and  a  set  of  measurable  (single-valued)  functions,  and  let 

^  :=  {if  :  if  :  R2  — >  K.  is  measurable  and  ip{x)  G  4'(a;),V  a;}. 

The  G-value  of  4f  is  defined  as  follows: 


G(\ 5)  :=  sup  inf  [  if(x)h{x)dx. 

hecg°-.f 

Theorem  4.14  [23]  Let  d\f\  denote  the  set-valued  sub -derivative  of  \f\,  i.e., 


d\f\(x) 


sign(/0))  f{x)  ±  0 

[-1,1]  /(*)=  0. 


Then,  for  the  TV+L 1  model  (14), 

(i)  u\  =  0  is  an  optimal  solution  if  and  only  if 

X~  G(d\f\)’ 

(ii)  u\  =  f  is  an  optimal  solution  if  and  only  if 


A  >  sup 
heBV 


ip/ii  -  \\m 

S\f~h\ 


(36) 

(37) 


Instead  of  directly  applying  Theorem  4.14  to  the  input  /,  we  apply  it  to  the 
indicator  functions  of  the  level  sets  of  /.  From  the  “layer  cake”  formula  (20),  it  is  easy 
to  see  that  solving  the  geometric  problem  (22)  is  equivalent  to  solving  the  TV+L1 
model  (14)  with  input  /  =  lr-  Therefore,  we  have  the  following  results  as  a  corollary 
of  Theorem  4.14: 


Corollary  4.15  For  the  geometric  problem  (22)  with  a  given  X, 
(i)  S \  =  (/}  is  an  optimal  solution  if  and  only  if 

X~G(d\lr\y 

(ii)  =  r  is  an  optimal  solution  if  and  only  if 


X  >  sup 
heBV 


IPlrll  ~  \\Dh || 

/  |lr  -  h\ 


The  above  corollary  characterizes  1e/  in  (35)  for  given  p  and  A.  Suppose  that 
the  set  S'  of  a  geometric  feature  coincides  with  U(f,p )  for  p  G  [po,p i).  Then,  for 
any  A  <  1/G(9|ls|),  S  is  not  observable  in  u\.  This  is  because  1/G(d\lu(f,^)\) 
is  increasing  in  p  and  therefore,  for  p  >  p0,  S f  ^  (and  hence  Is,  in  (35)) 
vanishes.  Once  A  >  1/G(9|ls|),  according  the  above  corollary,  1E  ^  0  for 
p  G  [po,pi),  which  implies  that  at  least  some  part  of  S  can  be  observed  in  u\.  For 
A  >  supfteW(||Dlr||-||L>/i||)//|lr-/i|,  we  get  S /iM  =  U(f,p)  =  S  for  all  p  G  [p0,pi) 
and  therefore,  the  feature  is  fully  contained  in  u\ ,  which  is  given  by  (35).  In  general, 
although  a  feature  is  often  different  from  its  vicinity  in  intensity,  it  cannot  monopolize 
a  level  set  of  the  input  /,  i.e.,  it  is  represented  by  an  isolated  set  in  U (/,  p),  for  some  p, 
which  also  contains  sets  representing  other  features.  Consequently,  u\  that  contains  a 
targeted  feature  may  also  contain  many  other  features.  However,  from  Theorem  4.12 
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and  Corollary  4.15,  we  can  easily  see  that  the  arguments  for  the  case  S  =U(f,  p)  still 
hold  for  the  case  S  C  U(f1p). 

Suppose  there  are  a  sequences  of  features  in  /  that  are  represented  by  sets 
,S'i .  S‘2,  ■  ■  ■ ,  Si  and  have  distinct  intensity  values.  Let 


Afin  := 


1 _  rax  ,=  qiin  \\DlsA\~\\Dh\\ 

^(^IlsJ)  ’  *  h£BV  f  1 1 5;  —  h\ 


(38) 


for  i  =  1, . . . ,  l.  If  the  features  have  decreasing  scales  and,  in  addition,  the  following 
holds 


^min  ^  ^max 


<  A™in 


<  A™ax 


<  . . .  <  A“in 


^  \  max 


(39) 


then  feature  i,  for  i  =  1, . . . ,  l,  can  be  precisely  retrieved  as  itAmax+e  —  u-Amin-e  (here  e 
is  a  small  scalar  that  forces  unique  solutions  because  A™  =  A™ax  is  allowed) .  This  is 
true  since  for  A  =  Af1111  —  e,  feature  i  completely  vanishes  in  u\,  but  for  A  =  A™ax  —  e, 
feature  i  is  fully  contained  in  u\  while  there  is  no  change  to  any  other  features. 


5.  Second-order  cone  programming  formulations 


In  this  section  we  show  how  to  formulate  as  SOCPs  the  discrete  versions  of  the 
three  TV-based  models  introduced  in  Section  3.  In  an  SOCP  the  vector  of  variables 
x  £  I"  is  composed  of  subvectors  xi  G  Kn;  -  i.e.,  x  =  (xi;  x2; . . . ;  xr)  -  where 
n  =  ni+ri2  +  ■  ■  ■  +  nr  and  each  subvector  Xi  must  lie  either  in  an  elementary  second- 
order  cone  of  dimension  rij 

JCni  =  {x;  =  (*°;xi)  £lx  K"i_1  |  ||xi||  <  x°}, 
or  an  fo-dimensional  rotated  second-order  cone 


Ui 

Qni  =  {x;  G  M”!  I  Xi  =  x,  2xi®2  >  Xi,x2  >  0}. 

i= 3 


(40) 


Note  Qni  is  an  elementary  second-order  cone  under  a  linear  transformation;  i.e., 
+  x 2^:  ~~  x 2^:  X3': ' ' '  ’  e  ^ ni  (Xl>  x 2’  •  •  • ;  xt)  G  Qni. 


With  these  definitions  an  SOCP  can  be  written  in  the  following  form  [2]: 
min  c^xi  +  •  •  •  +  CjTx,. 

s.t.  Aixi  +  •  •  •  +  Arxr  =  b  (41) 

Xi  G  /C”!  or  QUi,  for  i  =  1, . . .  ,r, 

where  Ci  G  U.ni  and  Aj  G  RmXTli,  for  i  =  1, . . . ,  r  and  b  G  Rm. 

Since  a  one-dimensional  second-order  cone  corresponds  to  a  semi-infinite  ray, 
SOCPs  can  accommodate  nonnegative  variables.  In  fact  if  all  cones  /C*  are  one¬ 
dimensional,  then  the  above  SOCP  is  just  a  standard  form  linear  program.  As  is 
the  case  for  linear  programs,  SOCPs  can  be  solved  in  polynomial  time  by  interior 
point  methods.  This  is  the  approach  that  we  take  to  solve  TV-based  cartoon-texture 
decomposition  models  in  this  paper. 
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In  practice  images  are  represented  as  2-dimensional  matrices,  whose  elements  give  the 
“grey”  values  of  corresponding  pixels.  In  this  paper  we  shall  restrict  our  discussion 
to  square  domains  and  hence  n  x  n  matrices  in  R2  for  the  sake  of  simplicity.  Letting 
fij,  Uij,  and  Vij  be,  respectively,  the  values  of  the  observed  image,  the  cartoon,  and 
the  texture/noise  at  pixel  (i,j),  we  have  the  following  relations: 

fi,j  =  Uij  +  Vij,  for  i,  j  =  1, . . . ,  n.  (42) 

In  all  three  models  considered  in  this  paper,  u  and  v  are  determined  by  minimizing 
the  total  variation,  TV(u),  to  regularize  u  subject  to  a  fidelity  constraint,  which  is 
different  in  each  model.  In  the  discrete  case  TV (u)  is  defined  by: 

TV(uf=  Y,  Ill'll,  (43) 

where  ||-||  denotes  the  Euclidean  norm,  i.e.,  HS+UjjH  =  (  ({d^u)ij)2  +  ((d^u)ij)2  )1/2, 
and  d+  denotes  the  discrete  differential  operator  defined  by 

d+ui/=  (  (dx u)i,j  -  ( dy  u)i,j  )  (44) 

where 

(<9+ for  i  =  1, . . .  ,n  -  1,  j  =  1, . . . ,  n,  ^ 
(9yu)i,j=uz,j+i  -  uitj,  for  i  =  1, . .  .,n,  j  =  1, . ..  ,n  -  1. 

In  addition,  the  differentials  on  the  image  edges,  for  j  =  1  and 

(dy  ii)itn,  for  i  =  1, . . . ,  n,  are  defined  to  be  zero. 

By  introducing  the  new  variables  tij  and  the  3-dimensional  second-order  cones 

(U,j  ;  {d+u)itj,  (dyU)ij)  G  /C3,  (46) 

for  each  pixel  (i,j),  i,j  =  1  we  can  express  min{TE(u)}  as  min{ Uj} 

subject  to  (46).  We  now  consider  ways  to  handle  the  non-TV  terms  in  models  (8), 
(10),  and  (14). 

The  linear  constraint  J  \f  —  u\  <  a  in  (14)  can  be  expressed  discretely  by  two 
linear  constraints: 

YWsj  -  ui,j)  <  a  (47) 

~  •Cbi)  <  (48) 

hi 

If  J  |/  —  u |  or  more  generally  say  min  \x\  appears  in  the  objective  of  a  minimization 
problem,  one  can  introduce  an  extra  variable  and  transform  min|a;|  into  equivalent 
problems: 


min  |cc| 

(49) 

4= 

min  t  s.t. 

x  <  t,  — x  <  t 

(50) 

/  def ,  \ 

(s  =  t  +  x)  4= 

min(.s  —  x) 

s.t.  2 x  <  s,  s  >  0. 

(51) 

Both  (50)  and  (51)  have  a  linear  objective  subject  to  linear  constraints.  In  some  cases, 
(51)  is  preferred  since  s  >  0  is  easier  to  handle  [19]. 
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In  Meyer’s  model  (8),  we  define  the  discretized  version  of  1 1  f  1 1  c?  <  c  as  the  infimum 
of 

over  all  51,32  G  satisfying  v  =  g±  +  dy  gi  using  forward  finite  differences. 

To  express  ||u||g  <  cr  (or,  equivalently,  \\\Zg\(i,j)  +  3l(b  j)lk°°  —  <J)  an  SOCP,  we 
introduce  a  3-dimensional  second-order  cone 

(9o(i,j)  ;  9i(i,j),92(i,j))  G  £3  (52) 

for  each  i,j:;  hence,  ||u||g  <  o'  if  and  only  if 

go(i,j)  <  O  (or  equivalently  go(i,j)  =  a),  for  all  i,j ,  (53) 

Next,  we  present  ways  to  express  the  two  penalty  terms  in  (10)  in  SOCPs. 
Using  forward  finite  difference,  the  residual  penalty  term  f  \f  —  u  —  d\g\  —  ^232 1 2 
is  implemented  discretely  as: 

X^l/-  u-d+gt  -d+g2\2.  (54) 

Clearly,  minimizing  (54)  is  equivalent  to  minimizing  si  subject  to  the  following 
constraints: 


2slS2  >  si  (55) 

s2  =  1/2,  (56) 

s3  =  s4,  (57) 

4  >  rh ■  (58) 


(56)  and  (57)  are  linear  constraints.  (55)  can  be  formulated  as  (si;S2;S3)  e  Q3  and 
(58)  can  be  formulated  as  (s4;  [rij]i<ij<n)  G  K,n  +1. 

For  the  penalty  term 


in  (10)  there  are  three  cases  to  consider.  When  p  =  1,  (59)  can  be  formulated  as 
the  sum  of  pgo(i,j)  over  all  i,j  =  1  ,...,n,  where  3o(*,i)  is  subject  to  (52).  When 
p  =  00,  the  case  is  equivalent  to  Meyer’s  model  and  hence  (59)  can  be  formulated  as 
/iff  subject  (52)  and  (53).  When  1  <  p  <  00,  we  use  second-order  cone  formulations 
presented  in  [2].  Let  us  consider  the  general  case,  the  p-norm  inequality 

/  n  \m/1 

(ENVro)  (6°) 

where  p  =  l/m  >  1  and  t  >  0.  If  we  introduce  s,;  >  0,  for  i  =  1 , ,n,  we  can  express 
(60)  as  the  following  set  of  inequalities: 

\xi\  <  Si  >0,  for  i  =  1, . . . ,  n 

n 

^  ^  si  —  ^5 
i= 1 


(61) 

(62) 
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which  is  equivalent  to 

Xi  <  ;  —  Xi  <  Si  >  0,  for  i  =  1,. . .  ,n  (63) 

n 

^2 Si  - 1 ’  (64) 

i— 1 

Let  us  now  illustrate  how  to  express  the  nontrivial  inequality  constraints  in  (63)  as  a 
set  of  3-dimensional  rotated  second-order  cones  and  linear  inequalities  by  a  concrete 
example.  Suppose  p  =  5/3,  i.e.,  m  =  3,  l  =  5.  Dropping  the  subscript  i  and 
introducing  a  scalar  z  >  0  such  that  z  +  x  >  0,  it  is  easy  to  verify  that  the  first 
inequality  in  (63)  is  equivalent  to  z  +  x  <  s3/5f2/5,  z  >  0  and  z  +  x  >  0,  which  in 
turn  is  equivalent  to  (z  +  x )8  <  s3t2(z  +  a:)3,  z  >0  and  z  +  x  >  0.  The  latter  can  be 
further  expressed  as  the  following  system  of  inequalities: 

w2  <  s(z  +  x),  w\  <  W\s,  w2  <  t(z  +  x),  (65) 

(z  +  x)2  <  W2W3,  z>  0,  z  +  x>  0,  (66) 

where  the  first  four  inequalities  form  four  rotated  second-order  cones.  The  same 
argument  applies  to  — x  <  s3^5t2^5  if  we  replace  x  wherever  it  appears  in  the  argument 
by  -  x . 

5.2.  SO  CP  model  formulations 

We  now  combine  the  SOCP  expressions  derived  in  the  last  section  to  give  SOCP 
formulations  for  (8),  (10),  and  (14). 

5.2.1.  Meyer’s  model 
min 

s.t.  Uij  +  Vi j  =  fij,  for  i,j  =  1, . . . ,  n 

(d£u)i,j  +  (ui+i,j  -  ui,j )  =  °>  for  i,j  =  1, . . . ,  n 

-(d+u)ij  +  {uitj+1  -  uitj)  =  0,  for  i,j  =  l,...,n  (67) 

Vij  -  {9i,i+i,j  ~  9i,i,j  +  92,i,j+i  -  92,i,j)  =0,  for  i,j  =  1, . . . ,  n 

9o,i,j  =  v,  for  i,j  =  l,...,n  +  l 

( ti,j  ;  (d+u)ij-,(d+u)ij)  G  K 3,  for  i,j  =  1, ...  ,n 

{9o,i,j  ;  9i,i,j,92,i,j)  e  1C3,  for  i,j  =  1, . . . ,  n  +  1, 

where  u,  v,  dfiu,  d+n,  g0,  gi ,  g2,  and  t  are  variables  and  /  and  <7  are  constants.  In 
addition,  we  define 

(<9+ u)nj  =  0,  for  j  =  1, . . . ,  n 
(<9+  u)i:n  =  0,  for  i  =  1, . . , ,  n, 

since  the  boundary  constraints  containing  them  also  contain  itn+ij’s  and  w^n+i’s, 
which  are  not  defined.  This  convention  also  applies  to  the  SOCP  formulations  of  the 
other  TV-based  models  in  this  paper.  Although  solving  for  u  and  v  is  our  ultimate 
goal,  they  can  be  canceled  in  the  above  formulation.  After  solving  the  problem,  v  can 
be  recovered  by  -  gi,ij  +  92,i,j+i  -  52, ip)  and  u  by  /  -  v. 
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5.2.2.  The  Vese-Osher  model  Conforming  to  [26],  we  only  give  the  SOCP  for  (10) 
with  p  =  1: 


inf  {  f  |  Vu|  dx  +  A 

u,9l,92  J 


|/ -  u  -  digi  -  d2g2\2dx  +  p,  J 


whose  SOCP  formulation  is 
min  Ud  +  Asi  +  h  Ei <i,j<n+i(wi,j  -  90,i,j) 


g\ 


dx} 


(68) 


'UiJ  ~ I-  Vi,j  ~~ l~~  ^i,j  —  ? 

for  i,j 

(d+  a)i,j  (r^i+ljj  Op )  —  0, 

for  i,j 

^)id  fo  (foji+l  Op )  —  0, 

for  i,j 

viJ  ~  (gi,i+ld  ~  9l,id  +  92,id+ 1 

-  92, i,j)  =  0, 

for  i,j 

S2  =  1/2,  S 3  =  S4, 

2 go, id  <  wi,j >  wid  V  o, 

for  i,j 

(fid  i  (d+u)id)  €  /C3, 

for  i,j 

(si;s2;s3)  €  Q3,  (54:0,1,0,2,.  ■ 

(go,i,j :  gi,i,ji  g2,i,j)  ^  ^  > 

r  ^  (p  vrn2 -\-l 

•  5  '  n,n /  ^  5 

for  i,j 

i  n 
,n 
> n 

,  n  (69) 

,  n  +  1 
i  n 

,  n  +  1, 


where  u,  v,  r,  d+u,  d+u,  go ,  g\,  <72 5  t,  w,  s i,  S2,  S3  and  S4  are  variables  and  /  is 
constant.  Similar  to  the  SOCP  for  Meyer’s  model,  u  and  v  can  be  canceled  in  the 
above  formulation  and  recovered  after  solving  the  problem.  Note  that  a  simpler  SOCP 
without  terms  that  contain  the  residual  (error)  r  can  be  derived  and  solved  for  more 
accurate  u  and  v.  The  original  VO  model,  however,  was  introduced  to  be  numerically 
solved  by  PDEs  and,  therefore,  must  have  the  residual  penalty  term  to  avoid  the 
equality  constraint  u  +  v  =  f. 

The  SOCPs  for  p  =  l/m  >  1  and  p  —  00  can  be  derived  using  the  techniques 
discussed  in  the  last  section. 


5.2.3.  The  TV+L 1  model 

min  Ei <<,,•<„  fij 


s.t.  -(d+u)ij  +  (Ui+ij  -  uitj)  =  0,  for  i,  j 
~(d+u)ij  +  (itij+i  -  Uij)  =  0,  for  i,j 

E i,j  >  Ei,j  fid  -  T  for  i,  j 

Ei  j  <  Eip  fid  +  T  for  i,  j 

( tid  ;  (d£u)id'>  ( dtu)id )  e  fc3!  for  bi 


1, . . .  ,n 
1,  •  •  ■  ,n 
1, . . .  ,n 

1. .  . .  ,n 

1.. ..,n, 


where  u,  v,  d+u,  d+u,  and  t  are  variables  and  /  and  a  are  constants. 


(70) 


5.3.  The  G -value 

We  note  that  the  SOCP  formulation  of  G(d\f\),  the  G- value  of  the  set- valued  function 
d\f\  discussed  in  Subsection  4.1,  can  be  obtained  by  solving  an  SOCP  that  is  a  simple 
extension  to  the  SOCP  formulation  of  Meyer’s  model.  For  supheBV  after 

homogenizing  the  objective  function,  its  SOCP  formulation  can  be  easily  developed 
based  on  the  SOCP  formulation  of  the  total  variation  term  J  \Dh\. 
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(d)  (e) 


Figure  1.  Inputs:  (a)  original  117  X  117  fingerprint,  (b)  original  512  X  512 
Barbara,  (c)  a  256  X  256  part  of  original  Barbara,  (d)  a  256  X  256  part  of  noisy 
Barbara  (std.=20),  (e)  original  256  X  256  4texture. 


6.  Numerical  results 

In  this  section,  we  present  numerical  results  for  the  three  cartoon-texture 
decomposition  models  and  compare  them. 

We  used  the  commercial  optimization  package  Mosek  as  our  SOCP  solver.  Mosek 
is  designed  to  solve  a  variety  of  large-scale  optimization  problems,  including  SOCPs. 
Before  solving  a  large-scale  problem,  Mosek  uses  a  presolver  to  remove  redundant 
constraints  and  variables  and  to  reorder  constraints,  to  speed  up  the  numerical  linear 
algebra  required  by  the  interior-point  SOCP  algorithm  that  it  uses.  We  could  also 
design  dedicated  reordering  algorithms  for  each  of  the  three  models  following  an 
approach  similar  to  the  one  described  in  [16]  to  lower  solution  times.  However,  this 
is  not  pursued  here  as  our  focus  is  on  decomposition  quality  comparisons.  In  the  first 
set  of  results,  we  applied  the  models  to  relatively  noise-free  images. 

Example  1:  In  our  first  test  we  applied  the  three  models  toall7xll7  fingerprint 
as  depicted  in  Figure  1  (a).  Figures  2  (a),  (b),  and  (c)  depict  the  decomposition  results 
given  by  applying  Meyer’s,  the  VO,  and  the  TV  +  L 1  models,  respectively.  The  left 
half  of  each  figure  gives  the  cartoon  part  u,  and  the  right  half  gives  the  texture  part 
v ,  which  is  more  important  in  fingerprint  extraction.  Since  VO  is  an  approximation 
to  Meyer,  they  produce  very  similar  results.  We  can  observe  in  Figures  2  (a)  and  (b) 
that  their  cartoon  parts  are  close  to  each  other,  but  slightly  different  from  the  cartoon 
in  Figure  2  (c).  The  texture  parts  of  (a)  and  (b)  appear  to  be  more  homogenous  than 
that  of  (c),  which  shows  a  whiter  background  near  the  center  of  the  finger.  However, 
in  (c)  edges  are  sharper  than  they  are  in  the  other  two  figures. 

Example  2:  We  next  tested  textile  texture  decomposition  by  applying  the  three 
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models  to  a  part  of  the  image  “Barbara”  as  depicted  in  Figure  1  (c).  The  full  Barbara 
image  is  depicted  in  Figure  1  (b).  Ideally,  only  the  table  texture  and  the  strips  on 
Barbara’s  clothes  should  be  extracted.  Surprisingly,  Meyer’s  method  did  not  give  good 
results  in  this  test  as  the  texture  v  output  clearly  contains  inhomogeneous  background, 
depicting  Barbara’s  right  arm  and  the  table  leg.  To  illustrate  this  effect,  we  used  a 
very  conservative  parameter  -  namely,  a  small  cr  -  in  Meyer’s  model.  The  outputs  are 
depicted  in  Figure  2  (d).  As  a  is  small,  some  table  cloth  and  clothes  textures  remain 
in  the  cartoon  u  part.  One  can  imagine  that  by  increasing  a  we  can  get  a  result 
with  less  texture  left  in  the  u  part,  but  with  more  inhomogeneous  background  left  in 
the  v  part.  While  Meyer’s  method  gave  unsatisfactory  results,  the  other  two  models 
gave  very  good  results  in  this  test  as  little  background  is  shown  in  Figure  2  (e)  and 
(f).  TV+L1  still  generated  a  little  sharper  cartoon  than  VO  in  this  test.  The  biggest 
difference,  however,  is  that  TV+L1  kept  most  brightness  changes  in  the  texture  part 
while  VO  keeps  them  in  the  cartoon  part.  In  the  top  right  regions  of  the  output 
images,  the  wrinkles  of  Barbara’s  clothes  are  shown  in  the  u  part  of  Figure  2  (e)  but 
in  the  v  part  of  (f).  This  shows  that  the  texture  extracted  by  TV+L1  has  a  wider 
dynamic  range. 

Example  3:  Finally,  we  applied  the  three  decompositions  models  to  4-textures 
depicted  in  Figure  1  (e).  Figures  2  (g)-(i)  demonstrate  the  differences  described  in 
last  paragraph  more  clearly.  The  texture  image  in  (g),  generated  by  Meyer’s  method, 
contains  more  large-scale  textures  (e.g.,  the  rope  nots  in  the  upper  left  part)  than  the 
texture  images  in  (h)  and  (i).  TV  +  LI  method  gave  the  texture  part  with  bigger 
contrast  as  depicted  in  (i),  in  which  we  can  clearly  see  the  rope  and  line  patterns  in 
addition  to  just  positions. 

In  the  second  set  of  results,  we  applied  the  three  models  to  the  image  “Barbara” 
after  adding  a  substantial  amount  of  Gaussian  noise  (standard  deviation  equal  to  20). 
The  resulting  noisy  image  is  depicted  in  Figure  1  (d).  All  the  three  models  removed 
the  noise  together  with  the  texture  from  /,  but  noticeably,  the  cartoon  parts  u  in  these 
results  (Figure  3  (a)-(c))  exhibit  a  staircase  effect  to  different  extents.  Compared  to 
the  parameters  used  in  the  three  models  for  decomposing  noiseless  images  in  the  last 
set  of  tests,  the  parameters  used  in  the  Meyer  and  VO  models  in  this  set  of  tests 
were  changed  due  to  the  increase  in  the  G-norm  of  the  texture/noise  part  v  that 
resulted  from  adding  the  noise.  However,  we  did  not  change  A  when  applying  the 
TV+T1  model  since  the  G-norm  of  signv  does  not  change  significantly.  In  subsequent 
tests,  we  used  an  increased  Lagrange  multiplier  p  and  A  when  applying  TV+T1  and 
VO  and  a  decreased  constraint  bound  er  when  applying  Meyer  in  order  to  keep  the 
cartoon  output  u  closer  to  /.  Nevertheless,  while  the  staircase  effect  remains  in  the 
cartoon  outputs  in  the  results,  noise  is  not  fully  removed.  To  summarize,  none  of  the 
three  decomposition  models  is  able  to  separate  image  texture  and  noise,  and  in  fact 
all  of  them  exhibit  staircase  effects  in  the  presence  of  noise  due  to  the  minimization 
of  TV(u). 

The  third  set  of  results  depicted  in  Figure  5  were  obtained  by  applying  the  TV+T1 
model  with  different  A’s  to  the  composite  input  image  depicted  in  Figure  4  (/).  Each 
of  the  five  components  in  this  composite  image  is  depicted  in  Figure  4  (S'i)-(S,5). 
They  are  the  image  features  that  we  are  interested  in  extracting  from  /.  We  name 
the  components  by  Si , . . . , S5  in  the  order  they  are  depicted  in  Figure  4.  Clearly, 
they  are  decreasing  in  scale.  This  is  further  shown  by  their  decreasing  G- values,  and 
hence,  their  increasing  Amin  values  (see  (38)),  which  are  given  in  Table  1.  We  note 
that  A™ax,  for  i  =  1,...,6,  are  large  since  the  components  do  not  possess  smooth 
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(c)  TV+L1  (A  =  0.4) 


(d)  Meyer  (a 


(h)  VO  (A  =  0.1  and  =  1.0) 


(i)  TV+L1  (A  =  0.8) 


Figure  2.  Cartoon-texture  decomposition  results:  left  halves  -  cartoon,  right 
halves  -  texture. 


edges  in  the  pixelized  images.  This  means  that  the  property  (39)  does  not  hold  for 
these  components,  so  using  the  lambda  values  Ai, . . . ,  Ag  given  in  Table  1  does  not 
necessarily  give  entire  feature  signal  in  the  output  u.  We  can  see  from  the  numerical 
results  depicted  in  Figure  5  that  we  are  able  to  produce  output  u  that  contains  only 
those  features  with  scales  larger  that  1  / A;  and  that  leaves,  in  v,  only  a  small  amount 
of  the  signal  of  these  features  near  non-smooth  edges.  For  example,  we  can  see  the 
white  boundary  of  S2  in  V3  and  four  white  pixels  corresponding  to  the  four  corners 
of  S3  in  V4  and  V5.  This  is  due  to  the  nonsmoothness  of  the  boundary  and  the  use 
of  finite  difference.  However,  we  can  see  that  the  numerical  results  closely  match  the 
analytic  results  given  in  Subsection  4.1.  By  forming  differences  between  the  outputs 
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(a)  Meyer  (a  =  20) 


(b)  VO  (A  =  0.1  and  /i  =  0.5) 


(c)  TV+L1  (A  =  0.8) 


Figure  3.  Cartoon-texture  decomposition  on  noisy  images:  left  halves  -  cartoon, 
right  halves  -  texture/noise. 


(S4)  (Ss)  (/) 

Figure  4.  (Si)-(Ss):  individual  feature  components  of  composite  image  (/). 


Mi, . . . ,  uq,  we  were  able  to  extract  individual  features  Si, . . . ,  S$  from  input  f.  These 
results  are  depicted  in  the  last  two  row  of  images  in  Figure  5. 
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(ll2—  Ul)  (U3—II2)  (ll4  -  M3) 


t 


(u5  —  u4)  (u6  —  u5) 


Fiernre  5.  TV+L1  deoomnosition  outmits  (ua  and  va  were  obtained  usiner  A 


Total  Variation  Image  Cartoon- Texture  Decomposition 


24 


Table  1.  G- values  and  Amin  of  feature  components  <Si, . . . ,  S&;  Ai, . . . ,  Xe  used 


to  get  u\. 

,  ...  ,UQ. 

81 

£2 

£3 

£4 

£5 

G- value 

19.39390 

13.39629 

7.958856 

4.570322 

2.345214 

Amin 

0.0515626 

0.0746475 

0.125646 

0.218803 

0.426400 

Ai  = 

A2  = 

^3  = 

A4  = 

V 
cn 

II 

V 
0 

II 

0.0515 

0.0746 

0.1256 

0.2188 

0.4263  0.6000 
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